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Abstract 

We prove that every two-player non-zero-sum Borel game with lower- 
semi-continuous payoffs admits a subgame-perfect e-equilibrium. This 
result complements Example 3 in Solan and Vieille (2003), which shows 
that a subgame-perfect e-equilibrium need not exists when the payoffs are 
not lower-semi-continuous. 

1 Introduction 

Borel games are sequential games where two players alternately choose actions. 
The payoff of each player is a function of the infinite sequence of actions that 
the players chose. Borel games were introduced by Gale and Stuart (1953), 
who studied zero-sum games where the payoff function is the indicator of some 
set. In other words, player 1 wins if the play generated by the players is in a 
given set of plays, and player 2 wins otherwise. Martin (1975) proved that if 
the winning set of player 1 is Borel measurable, then the game is determined: 
either player 1 has a winning strategy or player 2 has a winning strategy. This 
result implies that every two-player zero-sum Borel game has a value, provided 
the payoff function is bounded and measurable. 

Mertens and Neyman (see Mertens, 1987) used the existence of the value 
in multi-player non-zero-sum Borel games to prove that for every e > 0, every 
multi-player non-zero-sum Borel game has an e-equilibrium, provided the payoff 
functions are bounded and measurable. The e-equilibrium strategies constructed 
by Mertens and Neyman are as follows: each player i starts by following an |- 
optimal strategy in an auxiliary two-player zero-sum game Gi, where the payoff 
is that of player i, player i is the maximizer and the other players try to minimize 
player i's payoff. This goes on as long as no player deviates. Once some player, 
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say player i, deviates, the other players switch to an |-optimal strategy of the 
minimizcrs in the game Gi. 

Thus, the players start by generating a play that yields all of them a high 
payoff, and, if a player deviates, he is punished with a low payoff. This construc- 
tion has the disadvantage that in the punishment phase, the punishers may lower 
their own payoffs. Therefore, in real-life situations, players may be reluctant to 
follow the equilibrium strategies constructed by Mertens and Neyman. 

To deal with such non-credible threats of punishment, Selten (1965, 1973) 
introduced the concept of subgame-perfect equilibrium. A strategy vector is a 
subgame-perfect e-equilibrium if it induces an e-equilibrium after any possible 
finite history of actions. Ummels (2005) proved the existence of a subgame- 
perfect equilibrium when the payoff function of each player is the indicator of 
some set. His proof is based on the following recursive construction. First, one 
identifies all finite histories which are a winning position to either player 1 or 
player 2; that is, if this finite history occurs, one of the players can ensure that 
his payoff is 1. After each such finite history one instruct the winning player 
to play his winning strategy, and the other player is instructed to play his best 
response. One then identifies winning positions to the two players in the new 
game, assuming the behavior in the first set of winning positions is set. The 
process repeats itself, until it reaches a stable state. Ummels (2005) proves that 
the pair of strategies thus defined is a subgame-perfect equilibrium. 

In the present paper we show that every Borel game with bounded and lower- 
semi-continuous payoffs admits a subgame-perfect e-equilibrium, for every e > 0. 
This result complements Example 3 in Solan and Vieille (2003) that shows that 
in games with general payoff functions a subgame-perfect e-equilibrium need 
not exist. 

The determinacy of Borel games has attracted a lot of attention in descriptive 
set theory (see, e.g., Schilling and Vaught (1983) and Kechris (1995)). A rich 
literature identifies winning positions for the two players in the class of games 
that are played on graphs (see Gradel (2004) for a survey). Two-player zero- 
sum Borel games where used in the computer science literature to study reactive 
non-terminating programs (see, e.g., Thomas (2002)) and model checking in /x- 
calculus (see, e.g., Emerson et al. (2001)), and in economics to show that 
measurable tests are manipulable (Shmaya, 2008). 

Our result also relates to the game theoretic literature that studies the exis- 
tence of a subgame-perfect e-cquilibrium in various classes of infinite games, see, 
e.g., Solan and Vieille (2003), Mashiah-Yaakovi (2009) or Flesch et al. (2008, 
2009). In particular, our result generalizes results in Flesch et al. (2008, 2009). 

The paper is organized as follows. The model and the main result appear 
in Section [2] Section [3] contains the proof of the main result, and Section [4] 
contains some comments. 
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2 The Model and the Main Result 



Definition 1. A (two-player non-zero-sum) Borel game is a triplet (A^u 1 , it 2 ) 
where A is a set of actions, and u 1 ,u 2 : A™ — > R are payoff functions. 

The game is played as follows. At every odd stage t player 1 chooses an 
action at £ A, and at every even stage t player 2 chooses an action at £ A. 
While making his choice at stage t, the player knows the sequence {aj)j <t of 
actions that was chosen by the players in previous stages. The payoff to each 
player i is M l (ai,a2, . . .). The description of the game is common knowledge 
among the players. 

Comment 2. The assumptions that (1) there are only two players, (2) both 
players have the same action set, and (3) the set of action is the same in all 
stages, are made for simplicity of notations only. Nothing that is said below 
would be affected if there are more than two players, if the players have different 
action sets, or if the sets of actions depend on past choices of actions. 

Denote by the empty history at the beginning of the game. The set of 
possible historie^j] at stage t is H t = A* -1 . Denote by H 1 = UfceN^2fc-i and 
H 2 = U/teN ^2fe the sets of possible histories at decision points of player 1 and 
player 2 respectively. 

Definition 3. A strategy for player i is a function a 1 : H l — » A. 

We denote by E J the strategy space of player i. Every pair of strategies 
(a 1 , a 2 ) e E 1 x E 2 determines a unique play p = p(a 1 ,a 2 ) — (a t )teN £ ^4 N as 
follows: 

ax = <t 1 (0), (1) 
a-2k = c 2 (ai, a 2 , . • . , a 2 k-i), Vfc e N, (2) 
a2fe+i = cr 1 (ai,a 2 , ■ ■ ■ ,a 2 k), Vfc £ N. (3) 

We denote by ^(er^cr 2 ) = u l (p(a 1 ,a 2 )) the payoff of player i when the two 
players follow (a 1 , a 2 ). 

Definition 4. Let e > 0. A pair of strategies (c*,cr 2 ) is an e-equilibrium if 

uHala 2 ) > u\a\al)-e, Vct 1 £ E 1 , (4) 
u 2 (al,al) > u 2 (al,a 2 )-e, Vcr 2 £ E 2 . (5) 

Throughout the paper we endow A with the discrete topology, and A N with 
the product topology. 

The game is called zero-sum if u 1 ^) + u 2 (p) = for every p £ A N . The 
result of Martin (1975) implies that in zero-sum games, an e-equilibrium exists 
for every e > under merely a measurability condition. 

x By convention, A = {0} contains only the empty history. 
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Theorem 5. If the game is zero-sum, and if u 1 is bounded and Borel measur- 
able, then an e- equilibrium exists for every e > 0. 

This results implies the existence of an e-equilibrium in every two-player 
non-zero-sum game. 

Theorem 6 (Mertens, 1987). If u 1 and u 2 are bounded and Borel measurable, 
then an e-equilibrium exists for every e > 0. 

A stronger notion of equilibrium is the notion of subgame-perfect equilib- 
rium. Every finite history h — (a\, a 2 , . . . , at) G H 1 UH 2 , together with a pair of 
strategies (a 1 , a 2 ), determines an infinite play p = p(a 1 ,a 2 | h) = (b t )teN S A N 
as follows: 

bj = a,, 1 < J < Z, (6) 
b 2j - o- 2 (b u b 2 ,...,b 2j -i), K2j, (7) 
6jy+i - a 1 (b u b 2 ,...,b 2j ), K2j + l. (8) 

We denote by u % {o x ,o 2 \ h) = u 1 (p(cr 1 , a 2 \ h)) the payoff of player i at this 
play. 

Definition 7. Let e > 0. A pair of strategies {<y\,a 2 ) is a subgame-perfect 
e-equilibrium if for every finite history h G H 1 U H 2 one has: 

u\o-lal | h) > u\a\al \ h) - e, Va 1 G E 1 , (9) 
u 2 (al,al\h) > u 2 (g\,o 2 \ h) - e, Vcr 2 G S 2 . (10) 

Thus, every finite history h defines a subgame — the subgame that starts 
once the finite history h occurs. A strategy pair is a subgame-perfect e-equilibrium 
if it induces an e-equilibrium in all subgames. 

We say that a finite history h — {at)f— \ is a prefix of the play p — (bt)t£~N G 
A N , or that p is an extension of h, if a t = b t for every t S {1,2,..., I}, and we 
denote it by h -< p. We say that a finite history h — (a t ) l t=1 is a prefix of the 
finite history h! — (b t )^L 1 G A N , or that h! is an extension of h, if I < m and 
a t — b t for every t G {1,2,..., I}, and we denote it by h ^ h! . 

When A is endowed with the discrete topology, and A N is endowed with the 
product topology, then a sequence (p k )ke'N of plays converges to a limit p if and 
only if every prefix h of p is a prefix of all the plays (p fe )fcgN except possibly of 
finitely many of them. 

Definition 8. The payoff function u l is lower-semi-continuous if for every se- 
quence (p fc )fe £ N of infinite plays in H 1 U H 2 that converges to a limit p one 
has 

liminf u l {p k ) > u l (p). 

Note that every lower-semi-continuous function is Borel measurable. Our 
main result is the following. 

Theorem 9. If u 1 and u 2 are lower-semi-continuous and bounded, then the 
game admits a subgame-perfect e-equilibrium for every e > 0. 
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3 Proof 



We first note that if one changes the payoffs u 1 and u 2 by at most e (in the 
supremum norm) , then every subgame-perfect e-equilibrium in the original game 
is a subgame-perfect 3e-equilibrium in the new game. Because the range of the 
payoff functions is bounded, we can assume w.l.o.g. that the range of u 1 and 
u 2 is in fact finite. We will show that if the payoff functions are bounded and 
have finite range, there is a subgame-perfect O-equilibrium. 

For i G {1,2} we denote by —i the player who is not i. We denote by i(h) 
the player who has to choose an action after the history h; i(h) = 1 if h has 
even length, and i(h) — 2 otherwise. We denote by —i(h) the player who is not 
i(h). 

3.1 Subgame-perfect optimal strategies 

Definition 10. Let h G H 1 U H 2 be a finite history. The real number v(h) is 
called the value at h if 

v(h) = max min u^fer 1 , a 2 I h) = min max u^fcr 1 , a 2 I h). 

(11) 

Because the range of the functions u 1 and u is finite, the maximum and 
minimum in Ijlip are well defined. 

Definition 11. A strategy a\ G T, 1 of player i is called subgame-perfect optimal 
if for every h G H % one has 

u l {al,<j- 1 \h)>v(h), V^'er. 

The next result follows from Martin (1975). 

Theorem 12. In every zero-sum game with bounded and discrete payoffs, if u 1 
is Borel measurable then both players have subgame-perfect optimal strategies. 

3.2 Constructing some sequences 

In this subsection we define for every finite history h G H 1 U H 2 and every 
ordinal £, (a) a real number a^(h), and (b) a set H^(h) of plays. The sequence 
(ag (h))^ will be a non-decreasing sequence of lower bounds to subgame-perfect 
O-equilibrium payoff for the player who makes the decision at h, in the game 
that start at h. The sequence (H^(h))^ will be a non- increasing (by inclusion) 
sequence of sets of histories, such that all plays generated by a subgame-perfect 
O-equilibrium at the game that starts at h are included in every set in this 
sequence. 

Every play p — (a t )t£i<i G ^4 N defines a sequence (/i n )neN of finite histories, 
where h n = (a t )™ =1 is the prefix of length n of p. 

Suppose that one is given a real-valued function a that is defined over the 
set H 1 U H 2 of finite histories. 
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Definition 13. Let h be a finite history, and p a play that extends h. The 
play p is called a-monotonic at h if the two sequences (of(/i2fc)){fc: h^h 2k } an( ^ 
{a{h 2 k+i)){k-. h^h 2k+1 } are non- decreasing. 

A play p is called a-viable at a given finite history h if for each player i, the 
payoff u l (p) that p yields to player i is higher than a(h'), for every prefix h! of 
p that extends h and that is a decision history for player i. Formally!! 

Definition 14. Let a : H 1 U H 2 — > R be a real-valued function, and let h be a 
finite history. We say that a play p is a-viable at h if 

• h is a prefix of p, 

• u % ( h \p) > a(h') for every history h' € H 1 U H 2 that satisfies h <h' <p. 

The following lemma lists two simple properties of the play that is realized 
when both players follow subgame-perfect optimal strategies. It follows from 
the definitions, and its proof is omitted. 

Lemma 15. Fix a finite history h € H 1 UH 2 . Let a 1 and a 2 be subgame-perfect 
optimal strategies of the two players in the subgame that starts at h. Then the 
realized play ^(er 1 , a 2 ) is both cti-monotonic and ai-viable at h. 

Set 

ai(ft) = v l{h \h), (12) 
Hi(h) = {p e A N : h -< p,p is ai-viable at h}. (13) 

Thus, ai(h) is the value of the zero-sum subgame that "starts at ft" with payoffs 
that are the payoffs of player i(h). This is a lower bound on the equilibrium 
payoff of player i(h) in this subgame. Hi(h) is the set of all plays that yield 
both players at least their value in all subgames that may occur after ft. 

If h = (at) l t= i is a finite history with length I, and a £ A, we denote by 
(h, a) = [ax, a,2, ■ ■ ■ , en, a) the history of length / + 1 that starts with h and ends 
with a. 

For every successor ordinal £ + 1, define 

ct£ + i(h) = max min u ltyh \p), (14) 

aeA peH^(h,a) 

He+i{h) = { P eU aeA H s (h,a):u^ h Hp)>a (+1 (h)Y (15) 

If H^(h,a) contains all plays that can be generated by subgame-perfect 0- 
cquilibria in the subgame that starts at (h,a), then in every subgame-perfect 
0-equilibrium of the subgame that starts at h player i{h) will receive at least 
a^ + i{h). Moreover, after the history h, player i(h) will not play an action that 
does not maximize the right-hand side of (|14[) . and therefore H^ + i(h) contains 
all plays that can be generated by subgame-perfect 0-equilibria in the subgame 
that starts at h. 

2 This definition is adapted from Flesch et al. (2008). 



6 



For a limit ordinal £ define 

a^(h) — maxa\(h), (16) 

H^(h) = {p G A N : h < P,P is ^-viable at /i}. (17) 

The following observation, which follows from the definitions, will be used 
later. 

Lemma 16. Suppose that p G H^(h,a), where a G A achieves the maximum in 
the right-hand side of Then p G H^ + i(h). 

3.3 Properties of the sequences (a^(h))^ and (H^(h))^ 

The following theorem states some properties of the sequences (a^(h))^ and 
(H^(h))^, which play a crucial role in the proof of the main result. 

Theorem 17. The following holds for every h G H 1 U H 2 : 

1. The set H^(h) is not empty for every ordinal £. 

2. The sequence (H^(h))^ is monotonic non-increasing (by inclusion). 

3. The sequence (a^(h))^ is monotonic non- decreasing. 

4- For £ = 1 and for every limit ordinal there is a play p G H^(h) that is 
a^-monotonic at h. 

Proof. The proof is by transfinite induction. 

Part 1: H\(h) ^ for every h G H 1 U H 2 . Moreover, there is p G H\(h) that is 
a 1 -monotonic at h. 

This fact follows from Theorem [12] and Lemma [15] once both players fol- 
lows a subgame-perfect optimal strategy, the realized play is ai-viable and a±- 
monotonic at h. 

Part 2: a 2 (h) > ai(h) for every h G H 1 U H 2 . 

Every play in H\(h,a) is ai-viable at (h,a). Therefore 012(h) is the value 
of the game where at the second stage player i(h) receives the minimum payoff 
generated by an ai-viable plays. But ai(h) is the value of the game where at 
the second stage player i(h) receives the minimum payoff generated by a play 
which satisfies the ai-viability condition only for histories that are controlled 
by player i(h). This implies that indeed 012(h) > a\(h). 
Part 3: H 2 (h) C H x (h) for every h G H 1 U H 2 . 

This follows from Part 2 and Eq. fI5]). 
Part 4: If a (+1 (h) > a^(h) and H (+1 (h) C H^(h) for every h e H 1 U H 2 , then 
aa+ 2 (h) > ct£+i(h) and H i+2 (h) C H i+1 (h) for every h £ H 1 I) H 2 . 

This follows from the definitions (TT4"]) and (fT5|) . 
Part 5: For every limit ordinal £, every ordinal A < £, and every ft, G i? 1 U If 2 , 
one has a^(h) > ot\(h) and H^(h) C H\(h). 

This follows from the definitions (fl7j|) and IT7|) . 
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Part 6: For every ordinal £ and every h E H 1 UH 2 , If H^(h) ^ then H (+1 (h) ^ 
0. 

This follows from the definitions (fT4")l and (fT5|) . 
Part 7: For every limit ordinal £ and every h 6 -ff 1 U -ff 2 one has H^(h) ^ 0. 
Moreover, there is a play p € H^(h) that is a^-monotonic at ft. 

This is the difficult part of the proof. Fix a limit ordinal £ and a finite 
history h. We are going to generate a play that extends h, and we will show 
that it is in H^(h) and it is aj-monotonic at h. The play will be generated in 
iterations, where the construction in odd iterations differs from the construction 
in even iterations. We will then prove that an infinite play is generated after an 
even number of iterations. Finally we will prove that it is in H^(h) and that it 
is o^-monotonic at h. 
Odd iterations: 

Let hi be the history at the beginning of the iterations. For the first iteration, 
hi = h. For all other odd iterations, it is the history generated by the previous 
even iteration. 

Consider the following algorithm that generates a finite history or a play 
that extends hi. 

1. Let £i < £ be a successor ordinal that satisfies a^(hi) — a^ t (hi). Such an 
ordinal exists because every set of ordinals has a minimal element. 

2. Let ai be an action of player i(hi) that achieves the maximum in (|14p for 
hi and £i . 

3. Set hi = (hi, ai). 

4. Let £2 > ^i - 1 be the minimal ordinal that satisfies a^(h2) = ae^ih^). 
Note that because £ is a limit ordinal, £2 < 

5. Let <Z2 be an action of player that achieves the maximum in (|14p for 
h 2 and £ 2 ■ 

6. Set h% = (hi, 01,02). 

7. Continue this way to create a sequence (hi, £1, ai, /12, £2, ag, . . .) that ex- 
tends hi. The iteration ends when either £ m = I or £ m is a limit ordinal. 
If Cm > 1 is a successor ordinal for every m £ N, the iteration never ends. 

Note that because £ is a limit ordinal we necessarily have £ TO < £, for every 
m 6 N. 

The next lemma states that odd iterations are finite. 
Lemma 18. There is m 6 N suc/i i/iai either £ TO — 1 or £ TO is a limit ordinal. 

Proof. Assume that the algorithm never terminates: £ m > I is a successor 
ordinal for every m £ N, so that the algorithm generates an infinite sequence 
(hi,£i,ai,h 2 ,&,a2, . . .). 
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We first argue that for every to £ N one has 

H( m -i(h m+ {) 2 H£ m+2 _i(h m+3 ). (18) 

Note that because £ m and £ TO +2 are successor ordinals, the two ordinals £ m — 1 
and £ m +2 — 1 are well defined. Now, let p be any play in H^ iTi+2 ^i{h m+3 ) . By 
Lemma [T51 p G H^ m+2 (h rn+2 ). Because £ TO +i — 1 < £ m +2, by the induction 
hypothesis (Part 4), p G H^ m+1 -i(h m+2 ). By Lemma HH p G H^ m+1 (h m +i)- 
Because Cm - 1 < £,m+i, by the induction hypothesis (Part 4), p G i?^ m _i(/i m +i), 
as desired. 

Because both £ m and !; m+2 are successor ordinals, Eq. (fT4"]) implies that for 
every to G N 

(W)- (19) 

Because a{(/i m ) = a^ m (h m ) and a^(/i m +2) = a i m+ 2{hm+ 2 ) we deduce that for 
every to G N 

a^(h m ) < a s (h m+2 ). (20) 

Because the payoffs are discrete, the inequality a^(h m ) < a^(h m+2 ) can be 
strict only finitely many times. That is, there is M G N sufficiently large such 
that a^{h m ) = a^(h m + 2 ) for every to > M. Following the argument that Eq. 
(TST)]) holds we deduce that for each such to we have 

uz(h m ) = a im (h m ) = a im+1 -i(h m+2 ) = a^ m+2 (h m+2 ) = a^(h m+2 ), (21) 

so that £ m +2 = irn+i — 1- Because this equality holds for every m sufficiently 
large, and because there is no infinite decreasing sequence of ordinals, there is 
to such that either £ m = 1 or £ m is a limit ordinal, as desired. □ 

Even iterations: 

Let h\ be the history that was generated by the previous iterations. Then 
it is the output of the previous odd iteration. Denote by A the last ordinal £ m 
generated in the previous odd iteration. Then in particular either A = 1 or A is 
a limit ordinal, and A < £. Moreover, a^(hi) — a\(hi). 

By the induction hypothesis (Part 1 or Part 7), there is a play p G H\(h\) 
that is a,\-monotonic at h\. 

By the induction hypothesis (Parts 2, 3 and 4), a^(h') > a\{h!) for every 
prefix h! of p that extends h\. If a^(h') — a\(h!) for every prefix h' of p 
that extends hi, the even iteration is infinite. Otherwise, the output of the 
current even iteration is the shortest prefix h! of p that extends hi for which 
a^ti) > a x (h'). 

Because p G H\(hi), the play p is o^-viable at hi. Because a^(h') = ot\{h') 
for prefixes h' of p that extend hi, p satisfies the condition of c^-viability for all 
such prefixes. 

Lemma 19. Letp* be the play that we just generated. Thenp* is a^-monotonic 
at h. 
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Proof. For the part of the play added in odd iterations the monotonicity was 
proved in (|20|) . For the part of the play added in even iterations it follows from 
the construction. □ 



We are now ready to prove Part 7. 
Lemma 20. € H^(h). 

Proof. If the number of iterations if finite, so that the last even iteration is 
infinite, then the play p» is o^-viable at h. Indeed, by construction it is en- 
viable at ho, the history at the beginning of the last even iteration. The claim 
now follows from Lemma [T9l 

We now show that it cannot be that the number of iterations is infinite. 
Denote by (^n)neN ah finite prefixes of p*. Because is a^-monotonic at 
h, the two sequences (cx(h2k))h<h 2 k ana - ( a (f l 2k+i))h^.h 2k+1 are non-decreasing. 
Because the range of the payoffs is finite, these two sequences are eventually 
constant. At the beginning of an even iteration we have a^(hi) = a\{h\), and 
the sequence (ot\(h'))h> is non-decreasing in the part of p* that is added in an 
even iteration (for the definition of A, see the construction for even iterations). 
But a^(h') > ct\(h') for every prefix h! of p» that is added in the even iteration, 
so that a^(h') — a\(h') for every such hi . In particular, the even iteration does 
not end. □ 

□ 

By Theorem ^2) and Hp) it follows that 

Theorem 21. There is an ordinal such that a^(h) = ct£ m +i(h) and H^{h) = 
H^ + i{h) for every h £ H 1 U H 2 . 



3.4 Proof of Theorem M 

We now construct a pair (o-l,a 2 ) of strategies, and show that they form a 
subgame-perfect O-equilibrium. 

For every finite history h choose an -viable play p{h) that extends h and 
that satisfies 

tr*W(p(A)) = min u- i{h \p). (22) 

If player i deviates, and h is the finite history right after the deviation (so that 
i = —i(h)), then p(h) is an ct^-viable play at h that minimizes player's i's 
payoff. 

Let o\ be the following strategy: Follow the play p{%) as long as player 2 
follows p{%). Suppose that at stage 2k\ player 2 deviates from p{%). From stage 
2k\ + 1 and on follow the play p(/i2fci+i) as long as player 2 follows this play. 
Suppose that at stage 2fc 2 player 2 deviates from p{h2k 1 +i)- From stage 2fc 2 + 1 
and on follow the play p(h2k 2 +i) as l° n g as player 2 follows this play. Continue 
this way. 

The strategy a 2 of player 2 is defined symmetrically. 
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We now show that (al,a 2 ) is a subgame-perfect O-equilibrium. To this end 
we fix a finite history h £ H 1 and we show that 

u 2 (al,a 2 | h) < u 2 (al,al \ h), Va 2 e £ 2 . 

One can use similar argument to show that the analog inequality for player 1 
holds as well, so that (cr*,o- 2 ) is indeed a subgame-perfect O-equilibrium. 

Let a 2 G T, 2 be any strategy of player 2. Let p» = p{p\,o\ \ h) be the play 
induced by (<jI,<t 2 ) given h. This is the play that is generated if player 2 does 
not deviate. Let p = p{a\,a 2 \ h) be the play when player 2 deviates to a 2 . 

Denote by 2k\, 2&2, • ■ ■ the stages where a 2 and a 2 differ along p; in those 
stages player 1 observes the deviations of player 2. The sequence (2kj)j may be 
finite or infinite. Denote by pj — p{crl,cr 2 ) the play that player 1 start to follow 
at stage 2k j, for each j. 

We complete the proof by showing that 

u 2 (p) < u 2 ( Pr ). (23) 

It is sufficient to show that 

u 2 ( Pj )<u 2 (p*), Vj. (24) 

If a 2 and a 2 differ only finitely many times along p, Eq. (|2"3"|) follows from 
Eq. (|24p . If er 2 and a 2 differ infinitely many times along p, then the sequence 
(ft')iGN converges to p, so that Eq. ([23)1 follows from Eq. (|24|) and the lowcr- 
semi-continuity of u 2 . This is the only place in the proof where the lower-semi- 
continuity of the payoff functions is used. 

The proof of |24|) is by induction on j. Because p* is -viable at h, u 2 {p\) = 
(/i2fei ) < u 2 (p*)- For every j > 1, because pj is a^, -viable at h^ki > u2 (Pj+i) = 
ot£„(h2kj) < u2 {Pj), which is at most u 2 (p„) by the induction hypothesis. The 
proof is now complete. 

3.5 A Folk Theorem 

Our construction enables us to characterize the set of plays that can arise in a 
subgame-perfect O-equilibrium in the game with discrete payoffs. 

Theorem 22. A play p is induced by some subgame-perfect O-equilibrium if and 
only if p is -viable. 

Proof. If p is a^-viable, then the construction in Section 13.41 shows that it is 
the play that is induced by some subgame-perfect O-equilibrium. 

To see that the converse is true, we show that if (crj, a 2 ) is a subgame-perfect 
O-equilibrium, then p(cr*,<7 2 | h) is o^-viable at h, for every ordinal £ and every 
finite history h. 

Because ai(h) is the value of the subgame that starts at h, and because every 
subgame-perfect O-equilibrium is an equilibrium in the subgame that starts at 
h, the claim follows for £ = 1. 
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Suppose now that the claim holds for an ordinal £. Let h be any finite 
history. Because the claim holds for £, the play p(<rl,<j 2 | h) is a^-viable at 
(h, a) for every a, and therefore in any subgame-perfect O-equilibrium of the 
subgame that starts at h, the payoff to player i(h) is at least a^+i(h). This 
implies that p(o\, cr~ \ h) is a^+i-viable at h, for every h. 

Finally, the definition of for limit ordinals £ implies that if p(al,cr 2 \ h) 
is a \ -viable at h for every ordinal A < £, then it is also -viable at h. □ 

4 Comments 

4.1 Chance moves 

Borel games are deterministic, and the sequence of actions chosen by the players 
uniquely determines the outcome. In many situations there are chance moves 
along the game, where actions are chosen according to a known probability 
distribution. This situation is equivalent to the case where there is a third 
player who follows a specific non-deterministic strategy, whatever the other 
players play. Our proof can be adapted to this more general situation, and this 
will be done elsewhere. 

4.2 Positive recursive Borel games 

Recursive Borel games are games where some finite histories are terminating, in 
the sense that once they occur the payoff is determined (and the play that follows 
them does not affect the players' payoffs), and the payoff of every infinite (non- 
terminating) play is 0. Many positional games that are studied in the computer 
science literature have this form. The significance of this class of games to game 
theory was exhibited in the context of stochastic games by Vieille (2000), who 
used it as a step to proving the existence of an equilibrium payoff in every two- 
player stochastic game. A recursive Borel game is called positive if both u 1 and 
u 2 are positive functions. 

Flesch et al. (2008) studied positive recursive Borel game with finitely many 
states; these are positional games that are played on a finite directed graph, 
where each vertex is controlled by some player, and when the game reaches 
some vertex, the controlling player can choose whether to terminate the game, 
or whether to continue the game by choosing one of the edges that leaves the 
vertex. The terminal payoff depends only on the vertex where termination 
occurred, and not on the whole past play. 

Flesch et al. (2008) prove that every such game admits a subgame-perfect 0- 
equilibriumd In their proof, they define for every vertex s a sequence (ak(s))kew 
that is similar to our sequence (a^(h))^, they prove that this sequence is non- 
decreasing, and, because there are finitely many vertices, they deduce that there 
is fc* £ N such that ak._,+i(s) — ak t (s) for every vertex s. They then use a 

3 When transitions are random, Flesch et al. (2008) prove the existence of a subgame- 
perfect e-equilibrium, for every e > 0. 
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similar construction of the subgame-perfect O-equilibrium as the one that we 
used. 

In Borel games every history is a different vertex. Therefore one needs to 
employ a much more delicate construction, that differs from the one in Flesch 
et al. (2008) in two respects. First, when the number of vertices is infinite, 
there need not by G N such that ak,+i(s) — Uk,(s) for every vertex s, and 
therefore (a^(h))^ should be defined for every ordinal. Second, one also needs 
to take into account the possible plays, and introduce the sets (H^(h))^. In fact, 
this paper grew from an attempt to understand why the proof in Flesch ct al. 
(2008) fails for positive recursive games with infinitely many vertices. 

It turns out that for positive recursive Borel games the construction can 
be simplified, and a single odd iteration is sufficient to show that H^(h) is not 
empty for limit ordinals £. 

4.3 Borel games with general payoffs 

Example 3 in Solan and Vieille (2003) shows that without the condition that pay- 
offs are lower-semi-continuous, a subgame-perfect e-equilibrium need not exist. 
However, Solan and Vieille (2003) show that a subgame-perfect e-equilibrium 
does exist if one allows behavior strategies. The existence of a subgame-perfect 
e-equilibrium in behavior strategies was proved in other setups where the payoff 
functions are not lower-semi-continuous, see, e.g., Solan (2005) and Mashiah- 
Yaakovi (2008). 

In our proof the lower-semi-continuity of the payoff functions was used only 
in the last part, to show that any deviation a 2 that differs from a 2 infinitely 
many times cannot be profitable, as soon as any deviation a 2 that differs from 
a 2 finitely many times is not profitable. We do not know how the proof should 
be adapted to handle general payoff functions. 

In fact, the following example shows that our definition of and is not 
appropriate for general Borel games. Consider a Borel game with A = {a, b}. 
The payoff functions of the two players are as follows: 



Condition 


u\h) 


u 2 (h) 


Both players played b finitely many times 


2 


2 


Only player 1 played b finitely many times 


2 


1 


Only player 2 played b finitely many times 


1 


2 


No player played b finitely many times 









Note that u 1 and u 2 are not lower-semi-continuous. Playing b finitely many 
times is a dominant strategy for both players, so that the unique subgame- 
perfect 0-equilibrium payoff is (2,2). However, one can verify that cx^Qi) = 1 
for every finite history h and every ordinal £, so that the folk theorem, Theorem 
1221 does not hold, and our construction of the subgame-perfect 0-equilibrium in 
the proof of Theorem [T2l is invalid. 
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